Image and video data processing and manipulation have become increasingly challenging as the quality and volume of data have correspondingly increased, and further as the demand for accurate, presentable image and video has similarly increased. To this end, a wide variety of image sequence processing and analysis tasks have involved object-based video manipulation. However, many video objects, such as a person's face, a vehicle, or other objects in a scene may be only partially visible in respective frames in an image sequence. For example, scene boundaries, or occlusion by another video object and other image characteristics can create visibility issues.
Many approaches to processing image and video data have involved tracking objects in two-dimensional (2D) images, using a tracking algorithm-type approach to analyze video frames in order to estimate motion parameters and to follow an object's motion path. Such applications have involved both model-based features such as wire frame features, as well as region-based features such as points of interest on a calculated surface, region or active contours.
Despite these attempts at tracking objects, it has been difficult to associate target locations in consecutive video frames, especially when the objects are moving fast relative to the frame rate, are occluded, or move off the scene. In addition, many approaches to tracking objects have been limited in their capabilities, and often limited to tracking objects between frames (e.g., tracking frame-to-frame motion of a video object for video-object compression). These and other matters have posed challenges to processing image and video data.